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ABSTRACT 

/ The usefulness-^6f:xjwild:iple correlation as an 

extension of one .group evaluation designs was investigated. Program 
participants were measured on the aiaount of time each spent 
pe(rticipaj:ing in a specific activity.> This information, along with 
selected personal variables, was used in a correlational analy^sis- to 
determine the contribution of various program activities and personal 
variables to the measured changes on the criterion variables. It was 
found that multiple correlation can prbvide a wealth of information 
tc decision- makers, at .a. jnini^mua cost. . (Author/RC) 
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Many educational eval uatofs corttend that the ideal way td" evaluate 
or measure the impact of an educational program is to impose a rigid 
experimental design and control for as many extraneous and attribute 
variables as possible. The results of such a design allows evaluators 
to make more^ clearly statements concerning the effects of the educational 
program without the influence of competing conditfons. ; 

However ^i deal this classical experimental design might be, its use 
in evaluating, educational programs is not commonplace. Evaluators are 
constantly confronted with situations in which the use of rigid evaluation 
designs in the evaluation of educational programs is next to impossible. 
.Numerous reasons exist tur the sparce use of this ideal evaluation design^ 
bat probably the most important is the sopial and political problems 
encountered when attempting to randomly assign subjects to experimental 
and control groups. This problem has become parti cul^irly evident to 
eva-luators working with compensatory programs. Even in rare situations 
where a matched control group- can be obtained, the logistics of testing 
presents a major dbstacle along with tne lack of control over the control 
group, i.e., a control group subject attending a^ similaf type program. 
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Another major obstacle to employing elaborate experimental designs 
is the funding arrangements for evaluation. Agencies funding educational 
programs speak strongly about evaluation, but often fail 'to support their 
rhetoric-with adequate funds. 

J As a result of these and other problems,, a substantial number of 
educational, programs i§,,.util-i2ing other types o^.evaluation designs.^ 
An e/a'mtJl3ftion of evaluation reports submitted to various funding agencies 
over the past five years revealed a substantial number of educational 
programs hayve been evaluated utilizing a basi'c one group goal attainment 
model and evaluated utilizing a one group pretest-posttest design. ' In 
diagram form, the design can be seen as: ' 

T, . X T2 

Where: 

I 

= pretest ' , - , 

= posttest 
X = treatment (program) 
Utilizing this model, the evaluation plan would take measures keyed 
to program objectives at the beginning of the program, allow the program 
to progres;s, and post measure the oWectives at the close of the program. 
Program effectiveness, then, would be determined by the amount of T-j - Tg 
difference. 

With the severe limitations of this type of design, the best which can 
be generalized is that gains on the measured variables were made (or not 
made) and the results were statistically significant or not significant. 
The results of this type of design leaves unknown the effects of extratieous 
and. attribute variables on the performance of subjects or what aspects of \ 
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the program tended to contribute to the performance of the subjects. It 
is obvious that more information, than can be Vovided by a one group 
evaluation design, is needed for intelligent decisions to be made concerning 
the impact of educational programs. 

The purpose of this study was to determine the usefulness of multiple 
correlation as a technique for providing decision-makers .with additional 
information about educational programs evaluated by one-group procedures. 

Setting 

- ^ The setting used to test the usefulness of multiple correlation as 
a program evaluation technique was a federally funded prer school program 
entitled, "Responsive Environment Program for Spanish American Children 
(REPSAC)," located in Clovis, New Mexico.\ The program^, designed for 
low birth woighti educationa-l-Vy--handicajip.g4^J^^ 5-year-6lds, was 
designed to prpvide exper^iences aimed at developing, participants* intellectual 
ability, language ability in English, language ability in Spanish, sensory 
and perceptual discrimination, speech development, psychomotor development, 
self concept, and school readiness. Such experiences were provided with* ^ 
group and individualized or small group activities during a 3-hour day. 
The group activities were planned and administered using the "responsive 
environment concept" in a bilingual (Spanish-English) setting. The , 
individual or small group activities included: The Responsive Typing 
Booth, Project LIFE (Language Improvement to Facilitate Educafipn), 
Pi-aget Early Chiddhood Curriculum, Peabody Language Development Kit, 

♦Project funded by USOE, Bureau of Education for the Handicapped, Handicapped 
Children's Early Education Program, Grant No. OEG-0-73— 710, Project No. 
HOOIOSK. 



Language Master; and the Frostig Kit. ^ ' 

Due to a number cf circumstances, mentioned in the first part of 

^this paper, the use of a randomly assigned experimental -control group 

design was not possible. Circumstances also prevented the addition of 

any lype^ of > control group. Since the program's inceptiorTTn 1971, the 

prtmary^evaTuation, design 'used (with.^^he excepi>ion of two years when a 

jion-equij{alen£^onl;ro,l group was used) i&as a pfe^est-posttest one-group 

design* ' ' ' ' - 

The results of these .evaluation efforts have consistently revealed 

2 

a significant gain in all areas njeasured except intellectual ability. 
Although the results of the evaluation provided some information of a 
useful nature, the need existed for more specific information relating 
to the relationship between the various ^jo^m components and the 
performance of participants. Some indication of the contribution being 
made to participants* performance of specific instructional activities 
would provide additional information useful to the program staff. 
Information regarding the contribution of certain personal factors 
would also be. useful. ^ 

* » 

Procedure 

' Prior to the opening of school, the eva.luators and the progratn 
staff identified the specific structured program components whic^i were 
designed to help achieve a specific program objective. In addition-, 
personal factors were identified which could possibly be contributing^ 
to the performance of participants. This program analysis yielded ten 
program variables and six personal variables to be correlated with five 
of the seven posttest measures. These were: > i 



Rosttest Measures (Dependent Variables) 

Y-, ~ learning aptitude a ^ • 

= language ab,ijity in_Sp,anlsl]u— ^ 

= language ability in English 
Y^ = sensory and pefceptual discrimination 
Yg = school readiness 

Independent ' Var i abl es . • 

^ ^ . ^(f-^T>SaBoay^Kit (English) 

= Piaget Materials (English) / 

= Language Master (English) ^ ^ - 

Xg = Peabody Kit (Spanish) 
Xg = Piaget Materials (Spanish)^ 
Xy = Language Master (Spanish) 

Xg = Typing Booth ' ... ! 

Xg = Project LIFE • 
X.jQ= Frostig Kit 
X^.j= Art Activi'ties 
X^g"' Birth Weight 
X^3= Age 

X^ 4= HELPS Instrument (Henderson Environmental Process Scale) 
X.j5= Number of Siblings 
X^g= Education of Mother 
X^.7= Monthly Income of Parents 

The (*r. pendent variables were measured utilill.^. the following 
instruments: 



(1) Hiskey-Nebraska Test of Learni'og Altitude - Intellectual ability 

(2) Peabody Picture Vocabulary Test (English and Spanish) - English 
and SpamVh language a^ilj^ty.^ ^ 

(3) Frostig Developmental Test of Visual Perception - Sensory and 
Perceptual di^soriinirjcition 

"^'^ H)-'J!jci^enr^ for Disadvantaged Pre-School Children - 

School Readiness. J" . 

-Aj The independent variables ^ X^^ (program variables) were measured 
i/n terms of the amount of time each participant was exposed to the specific 

yinstructional activity. A time-log was developed and used by each teacher 
to record the time. The data for each program variable. represented the 
total amount of time each participant^ spent at the specific activity 
during the school year. 

Data for the independent variables birth weigl\t, age, number of 

-Siblings, education of mother, and monthly income were obtained from 
records kept by the schoolT Variable X^^, parental involvement, was 
measured by the Henderson -Environmental Process Scale (HELPS). 

At the end of the school year, posttests were administered both for 
the purpose of calculating :he pre and post differences and to be used 
as the dependent variable in the multiple correlational analysis. 

^ Data Analysis 

Data were analyzed with a basic linear regression model utilizing 

^ 3 

the SPSS program - Multiple Regression Analysis: Subprogram Regression. 

The-^SPSS program is a general statistical technique which analyzes 
the relationship between a dependent variable and a set of independent 
variables. Multiple regression can be viewed as either a descriptive . 



tool or as an inferential tool. Since the purpose of using this analysis 
was to describe the relationship between the post measures and the program 
and personal variables, conceptually, the technique was viewed as a 
descriptive tool . 

In general, the type of information sought from -this technique was: 

1. The overall relationship between each of the posvtest measures and 
the sixteen progmTand personal variables-. In addition^ to the overall 
r=elatiqhshi;gS53$lt|;;;;pec6entage of variance accounted for by the independent 

variables was'sougliv^ 

2. The interrelationships between the sixteen independent variables 
and the relationship between each independent variable and the dependent 

variabiles. ^..^ ... - a. , 

3. The combination of pr^pgram and; personal variables which account for 
the variance of the posttest measures and could, serve as best predictors 

of dependent variable performance. . , ; 

Results 

The technique produced the following Wormation concerning the impact 
of the sixteen program and personal variabTeU on participant performance-. 
Results are presented separately for each posttest measure. 
- 1. Learning aptitude. The sixteen vaViables correlated .67 with 
J^arjii^na aptitude posttest scores explaining 46 per cen^t pf the variance 
The thv^ee best predictor variables were: Peabody Kit (English), art f 
activities, and number of siblings in family. 

2. Language ability in Spanish. The sixteen variables correlated 
.93 with Spanish posttest scores explaining 86 per cent of the variance.* 

/ 

/ 

, /' 
/ 



The six best predictors were: Peabody Kit (English), age, education of 
mother, Typing Booth, Language Master, and monthly income of family. It 
is interesting to note that the individual correlation between the amount 
of time exposed to the Peabody Kit (Spanish) and performance- on the Spanish 
test was -.47. 

3. Language Ability in English.- The sixteen variables correlated 
.71 with English posttest scores explaining 50 per cent of the variance. 
The four best predictors were: Peabody Kit (English), ^Language Master, 
Typing Booth, and educa^tlon of mother. 

4. Sens6ry and perceptual discrimination. The sixteen variables 
correlated ,89 with Pros tig posttest scores explaining 79 per ce^it of the 
variance. The four best predictor variables were: age. Project LIFE, 
Typing Booth, and Frostig Kit. ^ / 

5. School readiness. The sixteen variables correlated .75 with 
school readiness posttest scores explaining 56 per cent of the variance. 
The four best predictors were: age. Typing Booth, art activities, and 
Project LIFE. 

Tables 1 and 2 are provided for the purpose of examination. Table 1. 

represents the individual correlations between ttfS program and persOjnaT 

1 

variables and the posttest scores. Table 2 presents the intercorrelations 
of the independent variables. 

Conclusion 

Multiple correlation as an extension of one group evaluation design 
appears to be a useful tool for providing additional information' about the 
impact of various program, activities and related factors on the performance 
of program participants. The information should prove valuable to program 
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planners and other decision 'makers with regard to activities that seem to 
be contributing and those that do not appear to be contributing. In this 
study, for instance, program planners have considerably more data to make 
program decisions than the pre and post analysis provided. 

The technique, obviously, is not yiithout its limitations, knong 
the most noticeable are the time and effort inv&lved in keeping the 
time-log, the difficulty in separating the structured experiences from 
the unstructured one§, and the necessity of computer use which may not • 
be available to some programs. 

In addition, using time as a unit of measure has its limitations. 
Most noticeably, a measure of 



It appears, however, that. 



:he quality**, of exposure time" was omitted, 
multiple correlation, can provide useful 



information about an educationiil, program. 



10 



O 



O 

U. 
QC 
US 

o. 

c/) 

LJ 

to 
o 



to 

O 

O 
< 
U. 



o 

CO 



to 

LJ 



O 



o 

Cu. 

to 
=> 
o 

a: 
< 

o 
J_ 

o 

LJ 

to 
o 
a. 

X 
LJ 

LJ 



5=C 
LJ 
UJ 

3: 

LJ 
00 

to 
o 



LJ 

o: 
o 



r— 

X 



><* 
in 

X 

X 

CO 
X 

i 

CM 
X 



X 



vo 

X 



X 



X 



CO 
X 



CM 
X 



ERLC 



V) to 

o tu 

Q.H- 



.}( 
•K. 
O 

in 
• 


CO 
CsJ 
• 

1 


1 


• 


-.05 


•K 
•K 
CO 


" ID 
_XV1 




do 
o 


CO 
CM 










• 


U3 
CM 


■K 

CO 

CO 


CM 
O 


CM 
CM 


o 


• 


• 

1 


• 


• 


• 

1 


CO 

o 


CO 


CM 


00 


o 


•K 

ID V 










CO 


•K 
■K 


o 


■K 
-K 
CO 
CO 


<• 




• 


• 


• 




CO 

o 


O 
CO 


1 

o 


ID 

CO 


1 




r 


1 


si 

1 


•K 

•K ^ 


•K 

U5 

CO 




cn 

CVJ 


•K 

ID 

CO 












1— 


o 


O 


ID 


CM 



(U ID 
•r- X 
t 

O CM 



4-> 



(U • 
O 



4J no 

C (U 

s. :p 
E ^ 

s. 

o 
+J 

QJ n3 

2 o 

SI Cr- 
S-\<U 



to 

S5' 




ID 

CO 



1^ 



o 



ID 

CO 



CO 



cn 
>- to 
o ^ 
o z 

CDcC 
ICQ. 
UJ to 



CO 
CM 



CM 



CM 



CO 
CO 



CM 



>- to 
o « 
O -J 

CO 

1x1 UJ 
CI. 



CM 



5;h 
CM 



CM 



c^ 



UJ 



o 



ID 
CM 



o 



CO 



CD 
LJ h« 

:><^ to 
to o 



CO 


*4J 


o 


O 


• 

t 






• • 






1— 


c 


CM 


0) 


• 


XJ\ 


1 


0) 














CM 




• 




1 






> 0) > 


LD 


Q) > QJ 




r— Q)r- 


« 




1 


ID r- 




O r-O 




• O O 


•K 


■ « 


CM 




CO 


j:: 0) 0) 


• 




1 














ft} (1} 


LD 


4J 4J 4J 




c c c 


• 


(t5 (t5 (t5 




U U U 












»|— 



(/) 

2 

U. 
II 

O 
r— 
X 



to 

(U 

> 

•4: 

u 
< 



X 



c 



cn 

J3 
4J 
S. 
•r- 

CO.,' 

tl ^ 

CM 
r— 
X 



C 
UJ 



4J 

o 

•Q 

ft} 

0) 
0. 

n 

CM 
X 



r— 

) 



4u 

0) 
D. 



(U 

ft} 



0) 

cn 

3 
cn 
c 
ft} 



X 



c c c 

Q) O) C7> 
'f— 'r-* 

to to to 

* -K -K 





(U 








0 












to 




Q. 


(U 




cn 




< 




II 


II 


CO 










X* 








• 




c 








Ol 




to 


• 




c 






(/) 




r— 


to 


ft) 







to 



11 

ID, 
X* 



4J 
O 
2: 

C 

o 

VO 
O 

•o 

UJ 
11 

X 



S 
o 
o 

c . 



>> 

r— 
C 

o 
11 

x" 



o 

0) 

a. 
>• 
11 

LD 
X 



s. 

o 
p 

s: 

p 

OJ 

cn 
ft} 

o. 

11 

X I 



, OL 

ito , 



1 5- 

\ (U 

• p 

1 rtJ 



(U 
cn 

cn 

-J 



P 
O 
O 

CO 

cn 
c 

•r- 

OL 



UJ 

u. 



p 
u 

O 

s. 
o. 



11 



X 



CO 
X 



cn 

X 



CO 
CO 

< 

Of 

< 




r- 

X- 

IT) 
>< 



O 



. a: 

CD 
O 

d: 

GU 

CM 

o 

a: 
CD r3» 
< - H- 

ID 

o: 

CO 

o 
o 

LU 

CO 

o 

X 

tu 



LU 
LU 



LU 
CD 



00 
O 



lU 
DC 

a: 
o 
o 

LU 



r* 
X 



CM 



X 



O 

>< 



to 

LU 0> 
^ X 
CO 
< 

o:: 

< CO 
> X 



X 



LO 
X 



in 

X 



X 



CO 
X 



X 



CM 



CM 



CM 



/ 

o 



CM 



CO 



CO 



O 



o 



in 



























t. 




O 


CM 


CO 




in 


CO' 




o 




* 


t 


* * 




* 




LD 


CO 


LD • 






CM 


O 






r— 


O 






• 














1 


J 







W 








%r 
















1 


1 




























■K 






cr> 


o 


O 


CO 


CO 


UD 






r— 


CO 


o 


CM 



r-i CM 



CM O 



o o 



CM rr» 

O' <^ 



cn CM 



CO " CM 
r- CO 



f— CO 
CM O 



cn cri 
1— o 



CM 
CM 




CM 
X 



CM 



CO 



in 

CO 



in 

CM 



r-H CM 

CM r- 



r— CO 



CO o 



o Ln 



4c 

CM 



CO 



^ - CM 
CM 



CO CM O 
O r— CO 



CO a\ 
CM CM 



LO CM 
CM O 



•K 
LO 

, m 



to 



CM 



7 ' 



i 



CM r— 

CM vCO- 



o 



I I \ I 

■K 

r- r*- 
O O ^ 



r— CM 

O ' CM 



O 



O 



CO ^ O 
f— O r— 



O 



CDs 



CM 



Kt a\ CTi CM 
I— r— r- CM O 



CO CM 
CO o 



$ 

CM r- 

in CM 



00 
CM 



o 



CO 



\ 



c 

CD 

4- 



Xi 

i2- 



«5> 

X 



X 



Ln 

X 



LO 00 
X ,X X 



O I— 
01 r— r-* 
XXX 



CM 



CO 
I— 
X 



X 



in 

r— 



LO 



X 



^n r- o 
o o o 



CU CUf 01 

x: x: x: 

4J 4** 4-> 

4J 4^ 4J 
n) r0 

4-> "P 4J 
C C C 
lO ro rts 
U V U 
•r» 

*H •r- *r- 

c: C C . 
cn D) D> 
r* ♦r* *r- 
CO 0%00 

^ f 



/ 



ERLC 



12 



4 



12 



NOTES 



,Sfie Chapter 9 in W. James PophRm, JglucationaV Evaluation , Etiglewood 
Cliffs: Prentice-Hall, Inc., 1975^, Al50 note references at the , 
end of chapter as additional sourges. 
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I 

Askins, Billy E., jst. al, . Responsive Environment Program for 
Spanish' American Childr en (REPSAC) : Third. Year Evaluation Study. 
Evaluation Report prepared for USOE (BEH) and the Clovis Municipal 
Schools, Clovis, New Mexico. (ERIC Document Reproduction Service 
No. ED 096 086.) 

\ ■ ^ 

Nie, Norman H., et. al. , Statistical Package for^ the Social Sciences 
New York: McGraw-Hill Book Company, 1975r See Chapten-20. 



